Using Information Retrieval to Evaluate Trustworthiness Assessment of Eshops

To protect consumers from online counterfeiting, several systems have been recently made available that check whether an ecommerce website is trustworthy or not. In this paper we study how to evaluate and compare trust checkers using an information retrieval methodology to gather suitable data and build a ground truth test collection. The main findings of our experimental evaluation are that the inter-checker agreement was moderate and that the single trust checkers presented relative advantages and disadvantages. Essentially, review-based systems were very precise but largely incomplete, whereas feature-based systems provided assessments for any ecommerce website being submitted but were more prone to errors.